Application of frames in Chebyshev and conjugate gradient methods

Richardson and Chebyshev Iterative Methods by Using G-frames

In this paper, we design some iterative schemes for solving operator equation $ Lu=f $, where $ L:Hrightarrow H $ is a bounded, invertible and self-adjoint operator on a separable Hilbert space $ H $. In this concern,  Richardson and Chebyshev iterative methods are two outstanding as well as long-standing ones. They can be implemented in different ways via different concepts.In this paper...

Multilevel Conjugate Gradient Methods

Extensions of the Hestenes-Stiefel and Polak-Ribiere-Polyak conjugate gradient methods with sufficient descent property

Using search directions of a recent class of three--term conjugate gradient methods, modified versions of the Hestenes-Stiefel and Polak-Ribiere-Polyak methods are proposed which satisfy the sufficient descent condition. The methods are shown to be globally convergent when the line search fulfills the (strong) Wolfe conditions. Numerical experiments are done on a set of CUTEr unconstrained opti...

Variable Metric Conjugate Gradient Methods

Towards Stochastic Conjugate Gradient Methods

The method of conjugate gradients provides a very effective way to optimize large, deterministic systems by gradient descent. In its standard form, however, it is not amenable to stochastic approximation of the gradient. Here we explore a number of ways to adopt ideas from conjugate gradient in the stochastic setting, using fast Hessian-vector products to obtain curvature information cheaply. I...

Conjugate Gradient Methods in Training Neural Networks

Training of artificial neural networks is normally a time consuming task due to iterative search imposed by the implicit nonlinearity of the network behavior. To tackle the supervised learning of multilayer feed forward neural networks, the backpropagation algorithm has been proven to be one of the most successful neural network algorithm. Although backpropagation training has proved to be effi...